Embodiments of the present invention relate to two- and three-dimensional environment user interfaces. More specifically, various embodiments of the present invention relate to user interface devices and methods allowing a user to intuitively navigate, mark and make selections.
With the wide-spread availability of computers in the later part of the twentieth century, animators began to rely upon computers to assist in the animation process. This included using computers to facilitate drawing-based animation, for example, by painting images, by generating in-between images (“tweening”), and the like. This also included using computers to augment physical animation techniques. For example, physical models could be represented by virtual models in computer memory, and manipulated.
A problem users encounter when producing animation and performing other complex tasks using a computer is how to efficiently and intuitively navigate, select objects and provide instructions. Typical user interfaces such as keyboards, mice, and the like often hinder the process.
In typical cases, the user may use a keyboard to enter numeric values to specify positions of objects in a scene in an animation system, for example. The problem with this is that users often do not have an intuitive feel of the correspondence between the numeric values and the object placement. Instead, the user has to view the 3D placement results on a display, modify the numeric values, view the modified 3D placement results on the display, etc. A similar problem is seen with slider-type graphical user interfaces on the display, where the user must move a slider, view the results, etc.
Another problem is that for highly complex rendering and modeling software programs, graphical user interfaces often consume a substantial portion of the computer display with icons, menus, toolbars, and the like. Accordingly, the actual workspace for the user for the three-dimensional environment may be disadvantageously small (e.g., 75% of the display, 50% of the display, or the like). This problem is not limited to animation systems.
In other cases, specialized user interface devices including multiple knobs, specialized joysticks including additional degrees of freedom, or the like may be provided. Drawbacks to such devices however, include that operation of such devices are still often non-intuitive and difficult for users to master. In addition to animation systems, such interfaces are useful for other skills.
Marking Menus and Multi-Touch
Marking menus are gesture-based radial menus that allow users to select a menu item by drawing a directional stroke. [Kurtenbach]. Multi-stroke marking menus extend the basic technique and allow users to efficiently traverse a hierarchy of submenus by drawing a sequence of directional strokes. [Zhao04]. Extensive studies have shown that users can draw such directional strokes quickly and accurately. [Kurtenbach94], [Moyle], [Zhao06].
Multi-touch input devices have recently become a popular alternative to both the mouse and the stylus, particularly for small-screen personal devices and for large-screen co-located collaborative work environments, such as Apple's iPod Touch/iPhone devices, Mitsubishi's DiamondTouch devices, Microsoft's Microsoft Surface devices, Perceptive Pixel Media Wall's devices, and Touchco's UnMousePad device. Unlike a mouse or a stylus, such multi-touch devices detect multiple points of contact and thereby support bimanual interactions. These devices have the potential to significantly increase the efficiency of interaction because users can overlap their hand motions and work with both hands in parallel. Yet, the kinematics of the hand also impose constraints on the range of motions different fingers can make. For example, on the iPod Touch/iPhone devices, it may be easier for the thumbs to make vertical strokes than outward horizontal strokes.
Hierarchical Marking Menus
[Kurtenbach], [Kurtenbach93], and [Kurtenbach94] describe marking menus and showed that these menus exhibit a number of desirable properties. Marking menus are scale-independent—the selection depends only on the orientation of the stroke, not on its length, and therefore users can efficiently draw short strokes with ballistic motions to select items. [Casalta]. Users can draw selection strokes in-place and do not have to make large round-trip traversals to select items from a fixed location menu. Moreover, users can draw the simple, straight-line strokes in an eyes-free manner without diverting attention from their primary task. Finally, marking menus provide a seamless novice-to-expert transition path as novices draw exactly the same selection strokes as experts.
However, a drawback of marking menus is that selection accuracy depends on menu breadth, or the number of items that appear in the menu. [Kurtenbach93] found that accuracy declines significantly when breadth is greater than eight. Compound-stroke (see, [Kurtenbach93]) and multi-stroke (see, [Zhao04]) marking menus allow for more menu items by allowing users to draw long zig-zag strokes or multiple strokes to traverse a hierarchy of marking menus. At breadth-8, however, these techniques perform well only up to depth-2 or depth-3 respectively. More recent techniques have used additional stroke attributes such as stroke position (see, [Zhao06]) and curvature (see, [Bailly08]) to further increase menu breadth. Wave menus (see, [Bailly07]) are a variant of multi-stroke marking menus designed to improve novice mode interaction.
Marking Menus on Touch Devices
Touchpads that can track a single point of contact have been commonplace on laptops for the last decade. [Balakrishnan98] shows an integrated touchpad with a mouse to allow the non-dominant hand to select commands using a compound-stroke marking menu. [Isokoski] shows a curved stroke variant of marking menus for entering numbers using a touchpad. They found that their curved strokes were more accurate but slower in movement time than drawing straight-line selection strokes. [Karlson] used directional strokes for thumb-based navigation on a PDA. More recently, [Yatani] used a combination of position and directional strokes to disambiguate the selection of closely packed items on a touch-based mobile device.
Bimanual Interaction Techniques
Guiard's Kinematic Chain Theory (see, [Guiard]) details the way hands work together to perform tasks in parallel. Many bimanual interaction techniques assign asymmetric roles to the hands where the non-dominant hand sets the frame of reference and the dominant hand performs fine-scale interactions within this reference frame. [Balakrishnan99], [Buxton86b], [Hinckley], [Kabbash]. Other techniques assign symmetric roles in which both hands perform similar actions. [Balakrishnan00], [Casalta], [Latulipe05], [Latulipe06], [Owen].
[Odell] presented an asymmetric bimanual marking menu technique in the context of a shape drawing system. The dominant hand selects a shape and the non-dominant hand selects a command to perform on this shape using a marking menu.
Controllers for console-based gaming systems, such as the XBox from Microsoft, usually include two joysticks, one for each hand. [Wilson] shows a two joystick based text-entry system using an onscreen keyboard and they have shown that such a symmetric bimanual approach is faster than using the default, single joystick technique. In other two-stick-based text entry systems, each stick operates a separate marking menu.
One-handed multi-stroke marking menus have proven to be effective for use with a stylus or mouse because they are scale-independent, in-place, eyes-free, and provide a seamless novice-to-expert transition (see previous section). However, improvements are still possible.